ABSTRACT
Introduction: Health information systems play an important role in improving the quality of patient care and patient safety. to ensure their effectiveness and efficiency, they need to be evaluated. Although HIS evaluation has been investigated in many studies, there is no consensus on which aspects of HIS to evaluate. The aim of this study is to identify the indicators for the evaluation of health information systems and to provide an overview of the criteria devised and studies conducted. Material and Methods: An umbrella review was performed exploring databases PubMed, Science direct, Web of Science, Science, and IEEE while following the PRISMA protocol. Articles were reviewed by two authors independently using the covidence tool to check the inclusion criteria and to extract the data items. Risk of bias was assessed using ROBIS and AMSTAR. Results: All included studies showed a high risk of bias according to ROBIS criteria. The extracted evaluation criteria were classified into 13 categories. Most of the studies believe that a more reliable and standardized tool is needed for the evaluation of health information systems. Two studies mentioned that surveys and questionnaires were the most commonly used method for evaluation of the systems. Summative evaluation was the most used method in two studies and the least used method in another one study. Conclusion: All the included studies had high risk of bias. Accordingly, further research and evidence is needed in this field. Most of the studies highlighted the need for more reliable and standardized tools for evaluation of health information systems. © 2022 Published by Frontiers in Health Informatics.